Japanese Publication Number P1997-193958A, for example, describes an invention related to a dispensing container with a pump. As shown diagrammatically in an example of this patent, such a dispensing container equipped with a pump is usually provided with a push-down head having an inside discharge port at the upper end of the pump to discharge the contents in the lateral direction. When the container is used, the push-down head is pushed down to make the pump piston descend. In many cases, the containers are shipped or stored with a separate head cover on the head from an appearance point of view or with a view to preventing the push-down head from being pushed in an erroneous or wrong operation.